Intelligent shared display infrastructure and associated methods

ABSTRACT

A two-tier display infrastructure switching architecture and associated method interconnects a plurality of network-enabled display devices. A video streaming switch is capable of receiving a video stream from a first network-enabled display device and forwarding the video stream to a second network-enabled display device. An access control service is capable of controlling access rights of each of the first and second network-enabled display devices to transfer video streams stream through the video streaming switch.

RELATED APPLICATIONS

This application claims priority to U.S. Patent Application Ser. No.62/001,483, titled “Intelligent Shared Display Infrastructure”, filedMay 21, 2014, and incorporated herein by reference.

BACKGROUND/PROBLEM TO BE SOLVED

Many conference rooms have shared displays that are manually configuredto show shared media. However, these shared displayed lack intelligence,configurability, and manageability.

SUMMARY

In one embodiment, a two-tier display infrastructure switchingarchitecture interconnects a plurality of network-enabled displaydevices. A video streaming switch is capable of receiving a video streamfrom a first network-enabled display device and forwarding the videostream to a second network-enabled display device. An access controlservice is capable of controlling transfer of the video stream throughthe video stream switch based upon access rights of each of the firstand second network-enabled display devices to use the video streamingswitch.

In another embodiment, a network-enabled display device with displayanalytic capture, includes a communication mechanism capable oftransferring video metadata in parallel with a video stream, a displayanalytic database for storing the metadata in relation to the videostream when transferred via the communication mechanism, and a displayanalytic communication module for providing access to the displayanalytic database by a third party.

In another embodiment, a method captures display analytic on anetwork-enabled display device. A number of users connected to thenetwork-enabled display device is captured over time and stored within adisplay analytic database. Display activity is measured over time basedupon one or more of: a number of different video streams transmitted tothe display device over time, a number of control signals transmitted toa display to modify its appearance and layout, and a number ofconnections and disconnection. Certain information from the analyticdatabase is reported on demand over a network.

In another embodiment, a method visually associates grouped videostreams on a network-enabled display device. Grouped video streams aredetermined based upon logical arrangement, by one or more users of thenetwork-enabled display device, of the video streams on thenetwork-enabled display device. The grouped video streams are displayedas a visual element on the network-enabled display device, and eachvideo stream within the group is collectively manipulated based uponmanipulations to the visual element by the one or more users.

In another embodiment, a method schedules availability of anetwork-enabled display device. A period defining intended use of thenetwork-enabled display device is received from a user. Informationrelated to one or more of display appearance, access rights, controlrights, media types allowed, and arrangement control of media posts isreceived from the user. Display specific visual resource descriptiondefining how the network-enabled display device presents media streamsreceived during the period is received from the user. The information isstored and an appearance and access rights of the network-enableddisplay device during the period is controlled based upon theinformation received from the user and the display specific visualresource description.

In another embodiment, a method indicates ownership of a video stream ona network-enabled display device. Ownership of each of a plurality ofvideo streams displayed on the network-enabled display device isdetermined based upon metadata of the video stream. An icon representingeach different owner of the video streams is displayed on thenetwork-enabled display device, and ownership of each displayed videostream owned by one owner corresponding to a selected one of the icon isgraphically indicated.

In another embodiment, a method controls display of video streams on anetwork-enabled display device by a first user authenticated as amoderator. A connection attempt from a second user of thenetwork-enabled display is received within the network-enabled displaydevice. Metadata associated with the second user is received within thenetwork-enabled display device. A connection request including themetadata is forwarded to the first user. In response to the connectionrequest and display of the metadata to the first user, an accept/rejectindication is received, and an accept connection signal is transmittedto the second user when the accept/reject indication indicates accept. Areject connection signal is transmitted to the second user when theaccept/reject indication indicates reject.

In another embodiment, a method arranges smart video streams on anetwork-enabled display device. Video and metadata network streams arereceived within the network-enabled display device. One or more of thevideo and metadata network streams is transmitted to a video arrangementprocessor. The video arrangement processor, implementing an algorithmthat automatically arranges display of the one or more video streams bythe network-enabled display device, is used to process the video andmetadata network streams to determine a stream transform, which is sentfrom the video arrangement processor to the network-enabled displaydevice. The position/scale/rotation and visual appearance of the videostream is modified within the network-enabled display device based onthe stream transform.

In another embodiment, a network-enabled display device has media filesharing capability. The display device includes a media sharingauthentication module having machine readable instructions that whenexecuted by a processor of the network-enabled display device arecapable of performing the steps of: receiving, within thenetwork-enabled display device, a media sharing request including anidentifier of a media stream and a target user; determining whether thetarget user can receive a media file associated with the media stream;and transferring the media file to the target user when the target usercan receive the media file.

In another embodiment, a network-enabled display device has proximateaudible discovery, and includes an audio protocol encoder for encoding aconnection information packet into an audio sequence; an audio processorfor driving a speaker to output the audio sequence as sound; and aconnection control module for controlling when the audio processoroutputs the audio sequence, such that the sound is received by amicrophone of a client device that (a) decodes the audio signal todetermine the connection information packet and (b) connects the clientdevice to the network-enabled display device based upon the connectioninformation packet.

In another embodiment, a method connects a client device to anetwork-enabled display device. An audio signal from the network-enableddisplay device is received within the client device via a microphonecoupled with the client device. The audio signal is decoded into aconnection information packet, and a network interface of the clientdevice is configured to (a) couple with a network identified within theconnection information packet and (b) to connect with thenetwork-enabled display device using network parameters defined withinthe connection information packet.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 depicts one exemplary intelligent shared display infrastructure,in an embodiment.

FIG. 2 depicts how a traditional network-enabled display device isaugmented for operation with the intelligent shared displayinfrastructure of FIG. 1, in an embodiment.

FIG. 3A shows one exemplary network-enabled display device augmentedwith analytics capture, in an embodiment.

FIG. 3B is a flowchart illustrating exemplary steps for display analyticcapture within the augmented network-enabled display device of FIG. 2,in an embodiment.

FIG. 4A shows images depicting a shared display that is displaying sixmedia posts, in an embodiment.

FIG. 4B depicts editing of a visual element that represents a group ofmedia posts, in an embodiment.

FIG. 4C depicts how a user may visually cycle the media posts within thevisual element of FIG. 4B, in an embodiment.

FIG. 5A is a schematic illustrating exemplary communication between anetworked-enabled display device and the scheduling system, in anembodiment.

FIG. 5B is a flowchart illustrating one exemplary method for schedulinga network-enabled display device, in an embodiment.

FIG. 6 depicts a traditional many-to-one video system that maps contentsources to positions on the display, in an embodiment.

FIG. 7 depicts one exemplary distributed content production and controlsystem, in an embodiment.

FIG. 8 shows exemplary owner-to-media ownership mapping, in anembodiment.

FIG. 9 is a flowchart illustrating one exemplary method for implementingcontrol of a connection event, in an embodiment.

FIG. 10A shows an overview of an exemplary stream mapping moduleillustrating interaction with a shared display, in an embodiment.

FIG. 10B is a flowchart illustrating one exemplary method for arrangingsmart video streams on a network-enabled display device, in anembodiment.

FIG. 11 is a schematic illustrating one exemplary network-enableddisplay device configured with a media sharing authentication modulethat facilitates ad hoc transfer of a streamed media file, in anembodiment.

FIGS. 12 and 13 show exemplary mechanisms and protocols for proximateaudible discovery and connection for a display device and a clientdevice, respectively, in an embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Each of the sections below covers a significant aspect of a softwaresystem that utilizes standard computing platforms, combined with variousdisplay devices (“displays”) to create a shared and accessibleinfrastructure of displays. Client displays connect to, share videoinformation, and control this display infrastructure, enabling multiplemedia streams (referred to as “posts”) on displays that are “published”from clients.

Two-Tier Display Infrastructure Switching Architecture for Multi-UserShared Displays

In the following description, a “network-enabled display device”includes a display, a processing unit, and a network interface. In oneembodiment, the network enabled display device is a flat-panel displayconnected to a networked computer via a video cable. In anotherembodiment, the network enabled display device is a smart-TV thatincludes an embedded processing unit. The network enabled display devicealso includes software-implemented algorithms that are executed by theprocessing unit to provide the functionality described herein.

The network-enabled display devices support video streaming from clientimage generators (hereinafter also called client devices) that candiscover and connect to them. Once connected, client devices are able topublish multiple video streams to the network enabled display device,sharing the display screen to collaboratively view image data. Systemsand methods described herein extend this sharing capability to atwo-tier display infrastructure that connects network enabled displaydevices to one another through a secondary video streaming service.

FIG. 1 is a high level schematic of one exemplary intelligent shareddisplay infrastructure 100. Client devices 102 connect to networkenabled display devices 104 through a local network. This localconnection carries video data, video metadata, and control packets.

At tier 1, client devices 102 connect to a network-enabled displaydevice 104 through a local area network connection, utilizing either aphysical network cable or wireless medium. The connection processutilizes a protocol that exchanges client information with networkenabled display device 104 to determine access rights, and transferother metadata including user name, and other client-specific metadata.Once connected, client devices 102 may stream media to network-enableddisplay device 104, and control the appearance of media displayed onnetwork-enabled display device 104.

At tier 2, a network-enabled display device video streaming switch(hereinafter “switchboard”) 106 and an access control service 108 form aglobal video streaming service 110 and cooperate to connectnetwork-display devices 104 together over both local area networks andwide-area networks. One or both of the switchboard 106 and the accesscontrol service 108 are implemented as machine readable instructionsthat are executed by a processor of a computer (e.g., a server) toimplement the functionality described herein. The access control service108 manages access rights of network-enabled display devices 104 toaccess the switchboard 106 and to stream video through the switchboard106. At the tier 2 level, network-enabled display devices 104 become“clients” of one another and are able to connect to othernetwork-enabled display devices, stream media, and control theappearance of media on other display devices.

With the intelligent shared display infrastructure 100, network-enableddisplay devices 104 are now able to connect to and communicate with aglobal service (i.e., streaming service 110) that supports wide-areaconnections. The switchboard 106 receives network enabled display deviceinput streams and forwards them to other network enabled display devices104 that act as subscribers. This intelligent shared displayinfrastructure 100 is unique in that client devices need not be aware ofthe switchboard and any potential complexities it could entail. Instead,network enabled display devices 104 continue to receive input signalsfrom clients 102 and then selectively forward those signals to theswitchboard 106 when appropriate. In this way, local area videoperformance is not impacted for users that are local to the display 104.When the need to transmit video to remote viewers arises, the networkenabled display device 104 either encodes its own display surface as anew video stream transmitted to the switchboard 106, transmits partialimage regions, and/or transmits independent posts already beingtransmitted by clients 102 connected to the display 104.

Network enabled display devices 104 support video streaming to displaysfrom client (image generator) devices 102 so that clients may connectand utilize display screens in an ad hoc manner. Streaming service 110extends this concept to become a potentially global infrastructure ofinterconnected displays 104.

Traditional video streaming systems support a single set of videosources that are broadcast/transmitted to a set of subscriber clients.An example of this type of video streaming system is a video streamingHTML server wherein web clients connect, subscribe and receive atransmitted video stream. In addition other architectures support apeer-to-peer mechanism that allows clients to receive video directlyfrom other clients. Neither of these prior art systems or architecturesmeets the needs of a distributed display infrastructure whose keycharacteristics are:

-   -   1) Direct, local network connection support between clients and        their displays    -   2) The ability of displays to communicate and stream video to        one another on-demand without impacting local network streaming        to either of the display endpoints

The present switchboard 106 organizes connections from multiplenetwork-enabled display devices 104 into two classes—producers andconsumers. Network enabled display devices 104 may become producersand/or consumers by contacting the switchboard 106 directly and thenrequesting to become either a producer or consumer. In some cases,displays need to have the appropriate credentials in order to become aproducer or consumer. In those cases, the displays contact the accesscontrol service 108 and request the appropriate credentials. Credentialsmay be granted or denied at this stage based on a number of factorsincluding whether the display 104 requesting credentials has paid asubscription, or if the display 104 requesting credentials for aparticular produced stream is presently over-subscribed.

The switchboard 106, then, is a central manager for a set of videoproducers and consumers (e.g., displays 104) that may be connectedtogether in order to support video transmission from any set ofendpoints (e.g., clients 102) to one another.

Switchboard Directory Service.

The switchboard 106 also contains a directory listing service databasethat encodes a unique identifier for each listed display. In addition tothe unique identifiers, the directory listing service encodes whichdisplays 104 are able to connect with one another. In addition thedirectory listing service encodes display class information that, inpart, determines which displays are able to connect, receive informationabout, or ignore one another.

In order for displays 104 to retain the many-to-one functionality of ourprevious inventions over a local network, but still allow those displaysto communicate over a wide area, they are augmented with several newcomponents. FIG. 2 depicts enhancements to network-enabled displaydevice 104 that facilitate use within infrastructure 100 of FIG. 1.

Components shown in red are used by a traditional network enableddisplay device for local media communication between client devices 102generating media streams and the display 204 itself. Incoming mediaposts from one or more client devices 102 are received by thenetwork-enabled display device 104 and converted to on-screen mediaposts M1, M2, M3 (See previous patents), shown as light blue on display204 in the FIG. 2.

Image generators connect to, post media, and control shared displays bycommunicating with the local network display module. For example, clientdevice 102(1) may connect to the display and post M1, and M2, followedby control signals to position M1 and M2 on the screen 204. In addition,each of these posts is associated with a video stream that is beinggenerated by the client device, so they are updated in real time overthe local network connection via a network display module 202.

The network display module 202 then determines an appropriate position,scale, and appearance of that stream and generates an image that will bedisplayed based on the multiple incoming media elements from severalclient devices 102 (in the example of FIG. 2, a second client device102(2) is creating a different video stream M3). The final compositedimage is sent to the physical display device for display.

Components that are needed to augment this model and interconnect twosuch shared displays are as follows:

A Switchboard Communications Module 210 is responsible fornon-video-based communications between the network-enabled displaydevice 104 and the switchboard 106 (not depicted in FIG. 2). Oneimportant role of module 210 is that it may request a connection toswitchboard 106 that may be authenticated if needed and rejected if thenetwork-enabled display device 104 does not have access to switchboard106. For example, a network subscription service may store anetwork-enabled display device name and its access rights to theswitchboard 106. If the network-enabled display device has not purchaseda subscription to the switchboard 106 and attempts to connect, theconnection may be rejected.

The switchboard communications module 210 may also connect to andcommunicate with the access control service 108 in order to receivecredentials that allow a network-enabled display device 104 to become aproducer or receiver at the switchboard 106. Once access rights havebeen granted the switchboard communications module 210 sets up acommunications with the switchboard 106 by exchanging information thatwill be needed to establish a communications channel between theswitchboard 106 and the network-enabled display device 104. Thiscommunications channel is used primarily for video transmission in thecase that the display 104 has connected to the switchboard 106 as aproducer or it will be used for receiving video in the case that thedisplay 104 has connected as a consumer.

The Switchboard Video Encoder 212 is used when a network-enabled displaydevice 104 becomes a provider to the switchboard 106. Switchboard videoencoder 212 is responsible for two tasks: 1) select and copy informationbeing displayed on the local shared screen 204; and 2) transmit a videostream to the switchboard 106 that contains the extracted video frames.The encoder 212 is able to, for example, select the top half of theshared display 204 and transmit it is a source video to the switchboard106. In addition to selecting and encoding a region of the shareddisplay 204, the encoder may simply select an incoming media source(e.g., M1) from a client device 102 and transmit the selected mediasource to the switchboard 106 as a source. This may involve transcodingthe video stream from the client device 102 but it may also simplyinvolve retransmission of the video stream to the switchboard 106.

The behavior of the video encoders 212 (i.e. which regions are selected,etc.) is controlled by requests from the content sources (e.g., clientdevice 102). For example, if a user wants to “mirror” a shared displayto a remote location, the user could, using a simple interface on one ofthe client devices 102 (image generators), a control interface on thedisplay 104 itself, or through a 3rd standalone display controller(e.g., a control panel in the room), select “mirror my display to theSan Francisco conference room”. The video encoder 212 would then selectthe entire screen and begin producing a video stream of the display tothe switchboard 106 as a new source.

The Switchboard Video Receiver 214 is used to receive an incoming videostream from the switchboard 106, decode it if necessary, and to providevideo frames to the local display 204. For each incoming video stream,the receiver/decoder 214 communicates with the shared display 204 toestablish a new media post (e.g., M1, M2, and M3). In this way, theswitchboard video receiver/decoder 214 acts as a virtual image generatoron the local network, sharing media to the shared display 204 in thesame way that a local client device 102 would.

Using the metadata associated with the incoming video stream from theswitchboard 106, information that distinguishes this video stream fromtruly local feeds may be used to augment the display 204.

Exemplary use of switchboard 106—Two users in a conference room aresharing both of their laptops to display D1 (e.g., display device 104).Media elements M1, and M2, corresponding to each of those laptops iscurrently arranged side-by-side on the display 204. User 1 decides to“call” a remote display.

-   -   1. The user clicks a user interface element that informs the        display 104 (and the switchboard modules 210, 212, and 214) that        a connection to the switchboard will be made.    -   2. The communications module 210 connects to the access control        service 108 and provides the credentials associated with a        license at D1.    -   3. The service 108 determines that D1 is able to utilize the        switchboard 106 to initiate calls and provides credentials for        the switchboard 106.    -   4. D1's communications module 210 transmits the access        credentials, its unique identifier, to the switchboard 106,        along with a request for an outgoing call to anyone it is able        to connect to.    -   5. The switchboard 106, using the switchboard directory service,        searches for D1 in the service directory and returns a remote        display list (unique identifiers, including intuitive names of        displays that may be connected to) of displays 104 that would        support a call from D1.    -   6. This information is received by D1s communication module 210.    -   7. This remote display list is then sent to the client device        102 connected to D1 that requested the call.    -   8. The user picks the destination display device 104 (e.g., “San        Francisco conference room”), referred to as D2 herein, from the        list. This selection is communicated to D1s communication module        210.    -   9. The communication module 210 then transmits a consumer        request with the identifier of the selected display to the        switchboard 106.    -   10. The switchboard 106 then transmits this request to the known        IP address of D2.    -   11. Either automatically, or through user input, this request is        either accepted or denied at D2.    -   12. D1 also transmits a producer request to the switchboard 106.    -   13. The switchboard video encoder 212 of D1 selects the screen        204 or a screen portion and begins transmitting the video to the        switchboard 106.    -   14. The consumer request, upon acceptance, results in        switchboard 106 forwarding the video stream from D1 to D2.    -   15. The D2 switchboard video receiver 214 establishes a        connection and posts a new media stream into the local screen        204—one that is the entire video stream from D1 in this example.

The above process is reversed from D2 to D1 so that a duplex channel ofvideo communications may be established. However, for full duplex, theincoming media stream from D2 is not selected by the encoder 212 at D1and is therefore not re-retransmitted to D2.

Other Cases that are Supported by this Architecture:

-   -   One display being “mirrored” to many displays    -   Two displays mirroring their outputs to one another (as media        posts)    -   Only one media post at a display being displayed on one or more        remote displays through the switch—the video encoder only        selects a single media stream from one image generator    -   The aggregate output of one display being transmitted and shown        on one or more remote displays as a single “media post”.    -   Many media posts being selected and retransmitted to remote        displays without encoding the full display.

System for Display Analytic Capture on Shared Displays

Traditional displays are connected to content sources through anexplicit video cable or video transport standard that is primarilyone-directional. Other than the setup stage, information outside of thevideo signal itself is typically not passed from the content source tothe display device. Even when information is passed from the display tothe content source that information is typically limited to scan linetiming, device capabilities, or other video-specific data. For example,in the case when a video cable acts as the physical medium, during thesetup stage an Extended Display Identification Data (EDID) structure ispassed from the display to the content source so that it can supply thecorrect video type based on the characteristics of the display that itis connected to. This information is limited and only carries data aboutthe device itself. In cases when a wireless or wired network is used toconnect a display to a content source—this information is stillrestricted to data related to the physical aspects of the displayitself. Because displays have, until recently, been viewed as only anoutput device that is driven by a single content source, video standardsand the systems that implement them have not considered more flexiblecommunication architectures that are full duplex. Furthermore, displaysare unable to store and analyze information beyond the video streamitself.

Intelligent shared display infrastructure 100 of FIG. 1 avoids the abovelimitations by introducing three components to the traditional contentsource->display architecture. When combined within infrastructure 100,these three distinctions create a novel display system that supports thecapture and analysis of display analytics in addition to simplydisplaying information provided by the content source.

First, a new communications mechanism is introduced that supports thetransfer of video metadata in parallel with the video stream itself.Examples of video metadata include the spatial location of the contentsource (e.g., client device 102), the login name of the user currentlyaccessing the content source, the length of the content that is beingstreamed, the revision history of the content, information about thecontent source platform (i.e. the operating system of the PC when thecontent source is a computer).

Secondly, a display analytics database module, associated with eachdisplay 104, is able to capture and store metadata related to incomingvideo stream connections from clients. This module, associated with thedisplay involves a database that stores events, metadata, andpotentially sections of the received video streams. In addition, thedata can be processed to derive store summary statistics, andautomatically associate metadata with video data.

Finally, a mechanism for accessing and displaying information that isstored in the display analytics database module is provided to supportthird-party access to the capture data. This mechanism is anetwork-enabled process that is able to connect to the display,selectively request access to the analytics data, and then display thatdata to a user.

FIG. 3A shows one exemplary network-enabled display device 104 augmentedwith analytics capture.

In one example of operation:

-   -   1) Client devices 102 connect to a network-enabled display        device 104 at periodic times to stream media, control media, and        exchange metadata.    -   2) Usage information from these activities are captured and        stored in a local usage database 302. Statistical information        (i.e. accumulated information about number of connections,        number of rejected connections, total bandwidth used, etc.) may        also be stored in the local usage database 302. A global        analytics communications module 310 may connect to the network        display device 104 through a display analytics communication        module 304 and request information that is stored in the local        usage database 302.-OR-the display analytics communication        module 304 may connect and transfer all or part of the        information stored in the local usage database 302 to global        analytics communication module 310. The global analytics        communication module 310 may communicate with any number of        these modules 304 (and display devices 104) on a network.    -   3) The information from one or more network-enable display        devices 104 is stored in a global usage database 312. Further        statistical information related to intra-display information        (i.e. the total bandwidth used for all displays) may also be        stored in the global usage database 312.    -   4) Information store within global usage database 312 may be        read and processed either through interactive queries, reports,        or rendered visually as graphical information through a display        analytics reporting module 314.

FIG. 3B is a flowchart illustrating the following steps for displayanalytic capture, in an embodiment. This flowchart implements thefollowing steps:

1) Capturing and storing the number of users connected to a display 104over time and then reporting that information on demand over a network.

-   -   2) Measuring display 104 activity over time using various        statistics: number of different video streams transmitted to the        display device 104 over time, number of control signals        transmitted to a display 104 to modify its appearance and        layout, number of connections and disconnection.    -   3) Number and reason for failed connections (i.e. exceeded user        limits)    -   4) Number and reason for failed new stream requests (i.e.        exceeded video bandwidth limits).    -   5) Capturing and storing information about the position (e.g.,        geographical location) of users.

Multi-User, Multi-Video Stream Visual Association Groups

The ability to “stack” many media streams together for user interactionand organizational purposes. Stacked media is marked as stacked togetherand then displayed as a single visual element along with other posts.

Stacks are:

-   -   created—by selecting a single stream that will be in the stack        or through some other UI element. Denoting media posts on a        shared display that should be put in a stack. For example, by        dragging a media post over a stack and dropping it into the        stack.    -   Modified—by editing the elements in the stack to order, delete,        remove, or add elements. Stacks have an implicit order as well        as a “focus element”.    -   Exploded—all elements in the stack are removed.    -   Ordered—elements in the stack can be reordered through user        interaction elements.    -   Cycled—the focus element is always rendered “on top” and visible        in the shared display, by cycling a stack the focus element may        be changed interactively.    -   Displayed—the stack element is rendered as a post on the shared        display. The focus element on the display is show on top.

Stacks are a feature that is valuable in the context of multi-sourcevideo streaming to a shared display. Stacking multiple live sourcesenhances organizational and visualization capabilities for the set ofusers trying to utilize the display for decision making andcollaboration. Prior art technologies for multi-source display andcontrol only support arrangements of the sources wherein a source iseither visible or not visible on the screen. When a source is visible itis typically arranged in a grid or other regular pattern based on thesize and number of other media source that share the display. Forexample, video wall controllers and video scalars are used to arrangemany sources onto a screen but the mapping between each source and itslocation/size on the screen is fixed and in a regular grid.

Stacks support logical arrangement of the video sources that are beingtransmitted to a display so that they may be visualized, grouped, andcontrolled, in ways that are visually connected.

Once grouped, stacks may be manipulated/arranged on a display 204 in asimilar way to other video streams. However, when a stack isrepositioned on a display (or scaled/rotated) the current members of thestack are updated accordingly.

FIG. 4A shows display 204 of network enabled display device 104 of FIGS.1 and 2 and/or augmented network-enabled display device 300 of FIG. 3Aillustrating a shared display with six media posts 402(1)-(6) (e.g., M1,M2, M3). Five of those posts 402(1)-(5) are a traditional video streamreceived from a content generator (e.g., client device 102) and thenpositioned on the shared display 204. Media post 402(6) is a visualelement that includes multiple media streams that have been organizedinto a “stack” that is shown on the shared screen 204 as a single post.A video of an ocean scene is the current focus of the stack and isdisplayed in the visual element location.

The visual appearance of stack 402(6) is augmented such that a viewer ofdisplay 204 understands that several video streams organized together.As shown in FIG. 4A, the stack post 402(6) is rendered to show multiplepost borders to give the appearance of a “stack”. In another embodiment,an additional icon on the group of video streams may be included todenote a group or stack of visual elements (e.g., video streams). Bymodifying the on screen parameters of the visual element, includingscale, position, rotation and other appearance modifiers, each elementwithin the stack inherits these changes and move, scale, and rotateaccordingly.

FIG. 4B depicts editing of a visual element 402(6) of FIG. 4A thatrepresents a group of media posts. In this example, visual element402(6) represents a stack containing 25 video streams that may bereordered or removed from the stack. In addition, users may add videostreams onto the stack 402(6) by dragging them into the stack editwindow (visual element) as shown. For example, a stack that contains the25 elements may also add a new video stream from a different imagegenerator simply by moving it into the visual element representing thestack.

FIG. 4C depicts how a user may visually cycle the media posts within thevisual element 402(6) of FIG. 4B. The current focus element is displayedin the foreground of the stack 402(6) and the sequential position ofnext 410 and/or previous 412 elements of the stack are also depicted inpart. A user may interactively provide commends to cycle throughelements in the stack such that the focus element changes and differentvideo streams of the stack are brought into view.

Stacks (e.g., stack 402(6) may be controlled in a distributed manner,any user/content source controller connected to the display may create,modify, delete, and cycle stacks through their own UI, when they haveaccess rights to do so.

Display Scheduling for Shared, Multi-User, Multi-Source Displays

Use of display device 104 of FIG. 1 may be scheduled. Typically,availability and use of prior art display devices was controlled byphysical connectivity. A user attaching a content source to a displaydevice defines the use of that device. Alternatively, access control(e.g., password/login) to functionality of a computer attached to thedisplay controls use of the display device. That is, prior art displayusage was defined by the physically interconnect between the displaydevice and the content source, or the display device utilization wasbased upon demand of users connecting to the display when it isavailable (i.e., when other users are not physically or otherwiseconnected to the display device).

In systems where wireless technologies are used to transport video tothe screen, users are able to connect on demand. Limitations toconnections are typically physical or only support more traditionalaccess rights such as user accounts. When using a display like a sharedresource, there are no ways to schedule and lock the display forspecific use.

Room schedulers schedule “meetings” that involve individuals, roomaccess, call setup for teleconferencing, etc. However, a room schedulerdoes not prevent a user from walking into the room and using, throughtraditional methods, a display device (e.g., display device 104) locatedwithin the room. For example, because video cables do not explicitlycarry information about access rights and scheduling, any user withaccess to a physical video cable connected to a display device maytypically access that display device irrespective of any schedulingcontrol.

A room scheduler may be improved to send scheduling information to thedisplay device 104, which allows the display device to: 1) controldisplay access for the network-enabled display device and 2) modify theappearance/behavior of the shared network-enabled display. With displayscheduling (also referred to herein as the scheduling system), displaydevice 104 and/or intelligent shared display infrastructure 100 includesan intelligent layer that is “aware” of scheduled events such that if agroup of users schedules the display device 104 as a resource at aparticular time, the network display module 202 within the displaydevice 104 will not receive connections from users outside of thescheduled group of users. Behavior change examples might be that a useris denoted as “host” in the scheduled meeting and this information isused to provision “moderator” rights to that user so he/she may controlthe shared display device 104, and accept/reject new media streams fromothers in the meeting. In addition, appearance on the display 204 maychange according to the scheduling. For example, when 30 seconds areleft in the scheduled meeting, a countdown timer may be displayed overthe streamed media and/or the media streamed from each user may begin to“Fade”. Display scheduling operates at a different level fromroom-scheduling systems.

Because this invention allows displays to be treated like shared andaccessible infrastructure, it is important that management tools areintroduced to allow users to access those displays at pre-determinedtimes.

FIG. 5A is a schematic 500 illustrating exemplary communication betweena networked-enabled display device 104 and a scheduling system 502.

The scheduling system 502 is in contact with one or more network-enableddisplay devices 104 that periodically update the scheduling system 502with availability information. Availability may also be provided to thescheduling system 502 through a 3^(rd) party module that maintainsdisplay availability through other means or through direct communicationwith the set of network enabled display devices 104. Alternatively,users may manually maintain display availability information by editinga database that contains information about display IP address, port,access rights, and times when particular sets of users are able toconnect to the display devices.

At scheduling time, users 504 or other modules are able to request 512 aset of available display devices 104. This request 510 may include timesthat a display device 104 is needed or other metadata related to thedisplay device (e.g., resolution, size). The scheduling system 502 thenresponds 514 with a set of available displays that satisfy the availabledisplay request. The user/scheduling agent 504 then transmits 516 aselection list to the scheduling system 502 that denotes which (set of)display(s) 104 is requested for scheduling. This information also maycontain details about which user may connect to the display deviceduring the scheduled time, what their roles may be (e.g., guest versuspresenter), and the scheduled time.

The schedule request is forwarded 518 to the network enabled display 104to ensure that the display device 104 is able to meet the schedule—forexample if the display device 104 has, in the meantime, been scheduledto shut down, or has lost its network connection, it may not beavailable to meet the scheduled request. In addition, information aboutthe schedule is stored at the network-enabled display device 104 topotentially modify the display 204 appearance and behavior of how mediastreams appear, which users may connect, and what control capabilitiesusers have.

Once a display device has acknowledged 520 the scheduling request, thescheduling service 502 updates its own information to reflect theschedule change. Finally, the scheduling service 502 notifies 522 theuser 504 that scheduled the display. Note that this notification mayalso be sent to more than just that one user, and could include amessage to other participants that were named in the scheduling requestfor example.

FIG. 5B is a flowchart illustrating one exemplary method 550 forscheduling a network-enabled display device, in an embodiment. Thisflowchart illustrates the following operational steps:

-   -   1) Users are presented with a list of displays that are        available (potentially filtered based on other mechanisms).    -   2) Users then select a display and determine a period of time        that the display will be used.    -   3) Information related to display appearance, access rights,        control rights, media types allowed, and even arrangement        control of media posts.    -   4) Display Specific visual resource descriptions can also be        defined. These descriptions may define how the display presents        the various media streams being transmitted to the display        during the meeting, how they are arranged for example based on        user or stream type.    -   5) This information is stored and then used to control an        appearance and access rights on that display for the scheduled        time.    -   6) Presenting, in addition to the various media streams that are        shown on the display, a countdown timer that denotes the end of        the meeting.    -   7) Presenting a name of the scheduled meeting on the shared        display.    -   8) Presenting a time to end of meeting, or meeting length as a        clock on the display.        User Ownership Mappings from Indicator to Media

For display systems that are able to display more than a single contentsource at a time, it can be important to know how the various mediastreams on the screen map to the various content sources. Displays ofthis type are typically connected to a video wall controller where themapping between content source and display position/size is determinedat setup time and, partly, defined by the physical interconnectionsbetween the content sources and the display.

FIG. 6 depicts a traditional prior-art many-to-one video system thatmaps content sources to positions on the display. This is accomplishedusing a layout and management controller module that informs avideo-switching module about how to map each of the input sources. Inthese systems, the size and position of each media a centralized layoutand management controller determines element positional mapping. Thiscontroller is an independent system that is operated or programmed tomodify the behavior of the switching module.

Traditional Many-to-One Video Switching System.

Limited number of sources identified at layout and management time.Centralized control of layout. There is little need to quickly identifythe correspondence between content source and position on the displaybecause these mappings are determined typically by a single sourcecontroller and they change rarely. When distributed control of mediamappings is possible, the mapping of content source to its appearance onthe display becomes potentially complex. Because we introduce anetwork-enabled display device that potentially may receive many mediastreams from a single source (a computer streaming multiple applicationwindows as well as a video file, for example) there is not a one-to-onemapping of image generator to video stream on the share display. What isneeded is a method to visually identify correspondence between the manymedia streams and their image generator sources.

FIG. 7 shows one exemplary distributed content production and controlsystem 700. FIG. 8 is an example of owner-to-media ownership mapping.FIGS. 7 and 8 are best viewed together with the following description.

Users may quickly reposition and scale media on display 204 at run-time,and therefore mapping between content source and its correspondingvisual appearance on the display 204 may be difficult to understand.Furthermore, a single content source may generate more than one post704. In the example of FIG. 7, a plurality of content sources702(1)-(N+1) (e.g., client devices 102) are shown. Content source 702(1)has connected to the display device 104 and is streaming two independentstreams to the display device 104 that are displayed as posts 704(1 a)and 704(1 b).

To allow users to understand the mapping between media source anddisplayed videos quickly and simply, distributed content production andcontrol system 700 visually denotes this mapping on the display 204itself. This comprises:

-   -   1) a visual representation being shown on the shared display of        the content source (see our Visual indicators of Presence . . .        patent). This is the content source icon 802.    -   2) a system that transports metadata in addition to the video        data that denotes ownership/origin of the video. This is the        ownership metadata    -   3) a display-side capability that reads that metadata and        maintains an ownership>video map. Media ownership database.    -   4) The visual indication of the ownership map on the shared        display. Ownership indicators 804.

Dynamic updates to ownership from new metadata messages from the contentsources or elsewhere should be covered (for example, a user “hands”control of a video source to a second source). The ownership indicatorscan be indicated a number of ways on the shared display. Examples are:

-   -   A set of lines 804 that connect the content source icon 802 to        each of the video sources (e.g., posts 704) owned by that        content source as currently indicated in the media ownership        database    -   Temporal animations that show ownership by moving media to        regions on the screen where the source icon is located based on        the currently indicated media ownership database

Content sources/users are able to activate, turn-on/off the visualindicators through interaction with their own controllers by sendingmessages to the network display module/display.

Visual ownership indicators (e.g., icons 802 and lines 804) may beactivated automatically when events occur. For example, when a newcontent source connects, when media layout controller modifies thearrangement of the sources on the shared display, when the there arechanges to the content source itself (i.e. a word document is edited ata content source that is content source streaming live).

Ownership indicators can also encode information about the owner. Forexample, a content source's user information, location can be encoded aspart of the visual ownership arrangement.

As shown in the example of FIG. 8, content sources (e.g., client devices102) are denoted by the set of icons 802 at the bottom of display 204and may each denote a user, her/his name, and so on. The three mediastreams being produced by that content source are denoted bylow-lighting (e.g., reducing intensity of) other elements (posts 704) onthe shared display 204, blurring those elements and then drawing lines804 from the icon 802 that denotes a content source to the posts 704themselves. As shown in FIG. 8, these visual effects, as well asanimations, may be used to quickly identify which content source (post704) is generated each of the video sources.

Accept/Reject Control with Metadata Previews

When a display 204 is being shared, it may be important that end usersare able to moderate both access and publishing rights. Display device104 differentiates users into two classes: guests and moderators. Atconnection time, display device 104 authenticates content sources (e.g.,client devices 102) into one of the two classes either through apassword or other mechanism. Note that it is possible to have a group ofmoderators who are moderating one or more guests.

Moderators are able to reject/accept connections from new contentsources that are in the guest category as well as accept/reject newmedia posts. In both cases the moderators are presented with metadataregarding either the connection request or the post so they are able tojudge whether to make an “accept” or “reject” response.

FIG. 9 is a flowchart illustrating one exemplary method 900 forimplementing control of a connection event.

Steps in a Connection Event.

-   -   1) One or more users connect to display device 104 and are        authenticated as a moderator.    -   2) A new connection attempt from a content source/user to        display device 104 occurs—and—the new user is unable to        authenticate as a moderator or opts out of the authentication        process.    -   3) Metadata related to the users information is transmitted to        the display device 104.    -   4) The display device 104 forwards a connection request data        stream to the moderator including the user metadata that was        received.    -   5) A dialog appears on the moderators user interface that alerts        the moderator that a new user is requesting a connection. This        dialog can include information in the metadata packet. Examples        of the metadata include the users name, connection request time,        operating system, how many connection attempts have occurred        over some time, the users geographic location, etc.    -   6) The moderator interactively selects from options that include        accept or reject.    -   7) This choice is transmitted to the display device 104.    -   8) The shared display then transmits an accept connection signal        to the guest users and a connection is established—or—the        display device 104 transmits a reject connection signal to the        guest.

The process described above is repeated when guest users are connectedto the shared display and attempt to post a new stream of media.However, in the case of a new media post, the moderator ultimatelyreceives, as part of the metadata packet, a preview image of the postthat the guest is attempting to post.

In the case that the media post is a video or live applicationwindow/desktop the media stream is transmitted to the display andforwarded to the moderator client in the form of a low frame-ratethumbnail video preview. This video preview for moderation in thecontext of a user who is moderating, dynamically, the ability of imagegenerators to create a new video stream to a share display device isunique. Once accepted, the video is then displayed on the shareddisplay.

In both cases, rejection by a moderator can be forwarded back to theguest client and notification of rejection occurs in a UI element thatcan include metadata about the moderator “Bob rejected your post”.

Self-Arranging Smart Video Streams on a Shared Display

In the case where users do not want to control layouts directly, orwould like to override their explicit control of media arrangements whenimportant information in the video streams themselves occur, displaydevice 104 includes a smart video streaming arrangement utility.

FIG. 10A is a schematic illustrating one exemplary stream mapping module1002 for automatically determining a target multi-video stream layout.Module 1002 utilizes the metadata and the video data being transmittedto the display device 104 to automatically determine a targetmulti-video stream layout for posts 704 on display 204. Stream mappingmodule 1002 receives video and metadata streams that are associated witheach media post on the display device 104, processes those streams, andthen transmits a stream transformation to the display device 104.Content sources (e.g., client devices 102), connected to the displaydevice 104, are already transmitting one or more of these video/metadatastream pairs. A video arrangement processor 1004 is responsible forcapturing these streams and processing the video data as well as themetadata in order to determine the correct transform to then transmit.This is accomplished by applying one or more of the Stream MappingModules 1002 to the incoming streams. Each stream mapping module 1002 isan algorithm that process streams and then determine a correspondingstream transform 1006(1)-(N).

Stream transforms 1006 are received by the network display module 202and then applied to the visual representation of that video stream onthe display 204. In this way the visual arrangement of the differentvideo streams on the shared display is impacted by the video contentitself.

One or more stream mapping modules 1002 may be associated with the videoarrangement processor 1004 and each module 1002 may be associated withzero or more of the content video/metadata stream pairs.

For example, a simple example of a stream mapping module would be tocompute the mean intensity in a video stream over time and then, basedon the video intensity output, producing a stream transform thatincludes a horizontal translation. The result of this would be toautomatically arrange “darker” videos to left of the shared display andbrighter images to the right.

Note that stream transforms may be geometric in nature but may includeanything that has to do with the visual appearance of the stream. Forexample, the transparency, display depth, if the video corresponding tothe stream is drawn with a border, or even if a stream is currentlyshown on the screen, may all be defined by the output stream transform.

A second example would be a stream mapping module 1002 that detectschanges in a video stream and produces a stream transform packet thatinforms the display device 104 to scale the video stream by 2× and todisplay that stream in front of other streams on the display 204. Inthis way a security system may automatically push an “important” streamof video to the foreground and grow it with respect to the other streamson the shared display 104.

Metadata may also be used to compute the output stream transform. Forexample, if a content generators metadata contains the geographiclocation of the client then the stream transform may automaticallyproduce a transform that arranges videos originating from clients in onepart of the world into a common area on the shared display 204.

Stream mapping modules 1002 may be activated/deactivated within thevideo arrangement processor 1004 by sending it control signals over thenetwork. For example, a user may connect to the arrangement processor1004, receive a list of the stream mapping modules that are active, andthen “turn off” one or more of the stream mapping modules 1002 on or offor modify which of the content sources are associated with which of themapping modules 1002.

FIG. 10B is a flowchart illustrating one exemplary method 1000 forarranging smart video streams on a network-enabled display device 1004.Method 1000 implements the following operation:

-   -   1) Sending video and metadata network streams to a shared        display 204.    -   2) Transmitting one or more of those same video and metadata        sources to a video arrangement processor 1004    -   3) Algorithmically processing stream data received by the video        arrangement processor 1004 to determine a stream transform    -   4) Transmitting the results of the stream transform to the        shared display 204    -   5) Modifying the position/scale/rotation and visual appearance        of the video stream based on the received stream transform.

Visual and Ad Hoc Peer-to-Peer File Relay System for Shared Displays

FIG. 11 is a schematic illustrating one exemplary network-enableddisplay device 104 configured with a media sharing authentication module1102 that facilitates ad hoc transfer of a media file 1104 that is beingstreamed to the display 204 for visualization purposes. Module 1102allows multiple client devices 102, connected to display device 104 overa network, to stream media files, application windows, or their entiredesktop, to the display 204, shown as media posts M1, M2 and M3 in thisexample.

In addition to media posts M1, M2 and M3, client connections arerepresented on the display 204 as a set of user icons 802(A)-(C). Theseuser icons 802 and media posts M1, M2 and M3 may be retransmitted to theclient devices 102 for display on their user interfaces.

An important prerequisite for this media sharing is that each mediastream being transmitted to the display device 104 from client devices102 is accompanied by metadata that encodes, at a minimum, ownershipinformation for each of the incoming media streams. This ownershipinformation includes the client network address and other informationthat is needed to establish a file transfer connection from the displaydevice 104 and the client device 102.

Where a media stream is denoted on a user interface of a client device102, the user may generate a media sharing request 1110 by clicking amedia stream and requesting a transfer. By interacting with the mediaposts M1, M2, and M3 and user icons 802 on the display device 104, auser may arbitrarily request a file transfer to one or many of the otherclient devices 102 (including the client initiating the request). Forexample, by right clicking media post M1, which is produced by clientdevice 102(1) from media file 1104, the user of client device 102(2) mayrequest that media file 1104 be transferred to client device 102(2).

In another example, the user of client device 102(2) may drag media postM1 to user icon 802(1) that represents client device 102(2). Thisinforms the shared display device through a set of control actions onthe media posts that the user of client device 102(2) is requestingtransfer of media file 1104 from client device 102(1) to client device102(2).

The media-sharing request 1110 includes an identifier of the mediastream(s) that are being requested, and the target set of client devices102. The media sharing request 1110 is processed by the media sharingauthentication module 1102. FIG. 11 is a schematic illustrating oneexemplary network-enabled display device with media file sharingcapability. Media sharing requests are held as “pending” by thenetwork-enabled display. A media sharing authentication module 1102,implemented as machine readable instructions that are executable by adigital processor of the network-enabled display device 102 for example,schedules a request to the user about the pending media and isresponsible for maintaining a pending queue 1106 of media sharingrequests 1110. The media sharing authentication module 1102 isresponsible for determining whether each media sharing request 1110 maybe fulfilled. For example, module 1102 may inspect the metadataassociated with the requested stream to determine (a) the sharing rightsassociated with the media stream, (b) whether the target client devices102 are able to receive media (users may, for example transmit a “blockall media files” message to inform the media sharing authenticationmodule 1102 that they reject any file transfer), (c) that sharing isenabled by the display device 104, and that the target user(s) areconnected to the display device 104.

A common approach may be to alert the target user(s). In this case, themedia sharing authentication module 1102 transmits a sharing request tothe target client devices 102 for approval/rejection by the user ofthose devices. When receiving this message, the client device 102 maydisplay an interactive UI element on the client interface that requestsapproval for receipt of the media file 1104. Upon approval, the mediafile 1104 is transmitted either directly to the target client devicefrom the source client device or relayed through the display device 104.

Source data and what is transferred on these requests may vary based onthe type of video stream that corresponds to the selected media post M1,M2 and M3. If the media post is streaming from client device 102(1) as amedia file 1104, then display device 104 may transport the media file1104 to the target client device 102(2). If the video stream beingtransmitted corresponds to an application window or the video desktop ofa client device 102(1), then an image file corresponding to desktopscreenshot at the time the request is generated is transferred to targetclient device 102(2).

Because the display device 104 is acting as a relay, it may augment thesource data based on information that is stored in the display device104 at the time of transfer. For example, the number of users connectedat the time, the arrangement of the various video streams at the time oftransfer on the display device 104 may be added to the transferred fileas one or more of images, video and metadata.

This interactive and authenticated file sharing process may also takeplace among more than two client devices 102 connected to the display.For example, suppose that two client devices 102(1) and 102(2) areconnected to the same display device 104 and are each streaming media asposts M1 and M2. A user of client device 102(3), also connected to thedisplay device 104 may indicate that post M1 should be shared withclient device 102(2) (for example, by dragging post M1 on display 204 touser icon 802(B). When this occurs, the authentication module 1102issues separate requests to each of client device 102(1) and 102(2). Theuser of client device 102(1), for example, who is generating post M1,may receive a request that includes information about the file transferrequest: user of client device 102(3) requests that you share mediastream M1 with the user of client device 102(2). At the same time, theuser of client device 102(2) may receive a similar request: user ofclient device 102(3) requests that you receive media post M1 from theuser of client device 102(1). Upon approval from both parties, transferof the media file is initiated.

Because the user of client device 102(1) knows the type and content filesource associated with M1, it may be transmitted to the display device104 and then forwarded to client device 102(2).

Proximate Audible Discovery and Connection Mechanism for Shared Displays

Prior art either makes use of a physical video cable that pre-definesthe mapping between a client image generator and a display or uses anetwork communications protocol to setup an ad hoc connection between aclient device and a display device.

Network communication methods focus on protocols that transmitconnection information from the display device to a set of potentialclient devices (e.g., image generators) so that the client devices may“discover” displays that are available for use. These protocols rangefrom a network broadcast of connection information packets where anydevice on the network may receive those packets, to more sophisticateddiscover mechanisms that support only point-to-point traffic but rely onnetwork routing tables and routing management steps.

The drawback of these approaches is twofold: 1) they rely either on aphysical video cable or a network connection, and therefore only whenthe client device and display are on the same network will they be ableto discover one another; and 2) since physical network connections mayspan arbitrary distances, these protocols to not inherently includespatial proximity. For example, when using a broadcast mechanism anydevice on the network that is shared with the display broadcasting it'sinformation may discover connection information, regardless of location.

FIGS. 12 and 13 show exemplary mechanisms and protocols for proximateaudible discovery and connection for a display device 104 and a clientdevice 102, respectively. FIGS. 12 and 13 are best viewed together withthe following description.

First, the shared display device 104 constructs a connection informationpacket 1202 that includes an identifying header that informs any readingdevice that the packet is a valid connection information packet. Thepacket also includes a display IP Address and connection Port number ofthe display device 104. Other network specific information may also beincluded in the packet such as the SSID of the network that the shareddisplay device 104 is connected to, and the IP address of the networkgateway. This packet may also include additional information aboutaccess information (e.g., whether the display device 104 requires apassword), whether the display device is currently accepting newconnections, how many client devices 102 are connected to the displaydevice 104, and so on. This connection information packet 1202 is thenencoded using an audio protocol encoder 1204 to generate a connectioninformation audio sequence 1206.

The audio protocol encoder 1204 maps data in the connection informationpacket 1202 to a series of audio frequencies, serialized over time. Thatis, each data element in the connection information packet 1202 becomesan audio frequency associated with a timestamp so that data at position1 in the connection information packet is associated with time t=0 anddata at position 2 is associated with time t=1. This process generatesthe connection information audio sequence 1206.

A connection control module 1210, associated with the shared displaydevice 104, is responsible for determining when this connectioninformation audio sequence 1206 is used, based upon user interface inputor programmatic control signals 1208, to generate connection informationaudio sequence and control/timing information 1212. The connectioninformation audio sequence and control/timing information 1212 is thenpassed to an audio processor 1214 that is attached to a speaker on theshared display device 104. The connection control module 1210 may beprogrammed to send the connection information audio sequence 1206 to theaudio processor 1214 at certain periodic time intervals or through userinput at the shared display device that requests it. When active andreceiving connection information audio sequence and control/timinginformation 1212 the audio processor 1214 drives the speaker to generatesound, resulting in an emitted audio signal 1216.

Within client device 102, a connection sequence listener control module1304 is activated via user interface input or programmatic controlsignals 1302 to generate audio receiver control 1306. The control module1304 may also be deactivated after a fixed amount of time or when userinput occurs.

When activated the connection sequence listener control module 1304generates audio receiver control 1306 to activate an audio receivermicrophone 1306 that generates a captured audio sequence 1310. Thecaptured audio sequence 1310 is input to an audio protocol decoder 1312.When the connection sequence listener control module deactivates itinstructs the microphone be disable and to no longer capture audiosequences.

The audio protocol decoder 1312 receives, within capture audio sequence1310, a time sequence of frequencies from the microphone and isresponsible for decoding those signals into a series of data elementsthat make up a received information packet 1314. This information packet1314 is processed by the audio protocol decoder 1312 to determine if theinformation packet 1314 contains valid connection information, such asby examining the header information of the received information packet1314. Invalid packets are ignored 1318.

Valid connection information packets 1314 are then transferred to animage generator of the client device 102 and used to establish aconnection between the client device 102 and the shared display device104.

The sequence of steps to establish a valid connection may involve firstconnecting the device to the correct network that the shared display isconnected to. A connection setup algorithm (disclosed previously) thenconnects the client display device to the IP address and port numbercontained in the connection information packet.

This connection process may be preceded by user interaction thatpresents information about the discovered display to a user operatingthe image generator client device. For example, when a valid connectioninformation packet is received by the client device 102, a userinterface dialog containing a list of one or more displays that areavailable connection may be presented to the user of the client device102. This list is augmented with information contained in the receivedinformation packet 1314, including an indication of whether the displaydevice 104 is locked, or in use by other image generators.

Once a connection between the client device 102 and the display device104 is established, the image generator within the client device 102then transmits a video stream and metadata stream to the shared displaydevice 104 to produce media posts (e.g., M1, M2, and M3 of FIG. 2) thatare displayed on the shared display 204.

Changes may be made in the above methods and systems without departingfrom the scope hereof. It should thus be noted that the matter containedin the above description or shown in the accompanying drawings should beinterpreted as illustrative and not in a limiting sense. The followingclaims are intended to cover all generic and specific features describedherein, as well as all statements of the scope of the present method andsystem, which, as a matter of language, might be said to falltherebetween.

What is claimed is:
 1. A two-tier display infrastructure switchingarchitecture for interconnecting a plurality of network-enabled displaydevices, comprising: a video streaming switch capable of receiving avideo stream from a first network-enabled display device and forwardingthe video stream to a second network-enabled display device; and anaccess control service capable of controlling transfer of the videostream through the video stream switch based upon access rights of eachof the first and second network-enabled display devices to use the videostreaming switch.
 2. A network-enabled display device with displayanalytic capture, comprising: a communication mechanism capable oftransferring video metadata in parallel with a video stream; a displayanalytic database for storing the metadata in relation to the videostream when transferred via the communication mechanism; and a displayanalytic communication module for providing access to the displayanalytic database by a third party.
 3. A method for display analyticcapture on a network-enabled display device, comprising: capturing, andstoring within a display analytic database, a number of users connectedto the network-enabled display device over time and then reporting thatinformation on demand over a network; measuring, and storing within thedisplay analytic database, display activity over time based upon one ormore of: a number of different video streams transmitted to the displaydevice over time, a number of control signals transmitted to a displayto modify its appearance and layout, and a number of connections anddisconnection; reporting, on demand, certain information from thedisplay analytic database over a network.
 4. The method of claim 3,further comprising the steps of: determining, and storing within thedisplay analytic database, information of a number and a reason for eachfailed connection; and determining, and storing within the displayanalytic database, information of a number and a reason for failed newstream requests.
 5. The method of claim 4, further comprising the stepof: capturing and storing, within the display analytic database,information about the geographical location of a user.
 6. A method forvisually associating grouped video streams on a network-enabled displaydevice, comprising: determining grouped video streams based upon logicalarrangement, by one or more users of the network-enabled display device,of the video streams on the network-enabled display device; displayingthe grouped video streams as a visual element on the network-enableddisplay device; and collectively manipulating each video stream withinthe group based upon manipulation of the visual element by the one ormore users.
 7. A method for scheduling availability of a network-enableddisplay device, comprising: receiving, from a user, a period definingintended use of the network-enabled display device; receiving, from theuser, information related to one or more of display appearance, accessrights, control rights, media types allowed, and arrangement control ofmedia posts; receiving, from the user, display specific visual resourcedescription defining how the network-enabled display device presentsmedia streams received during the period; and storing the informationand controlling an appearance and access rights of the network-enableddisplay device during the period based upon the information receivedfrom the user and the display specific visual resource description. 8.The method of claim 7, further comprising displaying, on thenetwork-enabled display device, a countdown timer that denotes the endof the period.
 9. The method of claim 7, further comprising displaying,on the network-enabled display device, a name of a scheduled meetingduring the period.
 10. The method of claim 7, further comprisingdisplaying, on the network-enabled display device, one of (a) a time toend of the period, and (b) a time since start of the period.
 11. Amethod for indicating ownership of a video stream on a network-enableddisplay device, comprising: determining ownership of each of a pluralityof video streams displayed on the network-enabled display device basedupon metadata of the video stream; displaying, on the network-enableddisplay device, an icon representing each different owner of the videostreams; graphically indicating ownership of displayed video streamsowned by one owner corresponding to a selected one of the icons.
 12. Themethod of claim 11, further comprising reducing illumination intensityof video streams owned by owners other than the one owner.
 13. Themethod of claim 11, wherein the steps of determining and indicating areactivated automatically based upon one of (a) video stream connection,(b) modified arrangement of the displayed video streams, and (c) changesto the video stream source.
 14. A method for controlling display ofvideo streams on a network-enabled display device by a first userauthenticated as a moderator, comprising: receiving, within thenetwork-enabled display device, a connection attempt from a second userof the network-enabled display; receiving, within the network-enableddisplay device, metadata associated with the second user; forwarding aconnection request including the metadata to the first user; receiving,in response to the connection request and display of the metadata to thefirst user, an accept/reject indication; and transmitting an acceptconnection signal to the second user when the accept/reject indicationindicates accept; and transmitting a reject connection signal to thesecond user when the accept/reject indication indicates reject.
 15. Amethod for arranging smart video streams on a network-enabled displaydevice, comprising: receiving video and metadata network streams withinthe network-enabled display device; transmitting one or more of thevideo and metadata network streams to a video arrangement processor;processing, using an algorithm implemented by the video arrangementprocessor to automatically arrange display of the one or more videostreams by the network-enabled display device, the video and metadatanetwork streams to determine a stream transform; sending the streamtransform from the video arrangement processor to the network-enableddisplay device; and modifying, within the network-enabled displaydevice, the position/scale/rotation and visual appearance of the videostream based on the stream transform.
 16. A network-enabled displaydevice with media file sharing capability, comprising: a media sharingauthentication module having machine readable instructions that whenexecuted by a processor of the network-enabled display device arecapable of performing the steps of: receiving, within thenetwork-enabled display device, a media sharing request including anidentifier of a media stream and a target user; determining whether thetarget user can receive a media file associated with the media stream;and transferring the media file to the target user when the target usercan receive the media file.
 17. A network-enabled display device with aproximate audible discovery, comprising: an audio protocol encoder forencoding a connection information packet into an audio sequence; anaudio processor for driving a speaker to output the audio sequence assound; and a connection control module for controlling when the audioprocessor outputs the audio sequence; wherein the sound is received by amicrophone of a client device that (a) decodes the audio signal todetermine the connection information packet and (b) connects the clientdevice to the network-enabled display device based upon the connectioninformation packet.
 18. A method for connecting a client device to anetwork-enabled display device, comprising: receiving, within the clientdevice, an audio signal from the network-enabled display device and viaa microphone coupled with the client device; decoding the audio signalinto a connection information packet; and configuring a networkinterface of the client device to (a) couple with a network identifiedwithin the connection information packet and (b) to connect with thenetwork-enabled display device using network parameters defined withinthe connection information packet.